Theory of quasiparticle interference on the surface of a strong topological insulator 
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Electrons on the surface of a strong topological insulator, such as Bi2Te3 or Bii-a^Sbcc, form a 
topologically protected helical liquid whose excitation spectrum contains an odd number of massless 
Dirac fermions. A theoretical survey and classification is given of the universal features, observable 
by the ordinary and spin-polarized scanning tunneling spectroscopy, in the interference patterns 
resulting from the quasiparticle scattering by magnetic and non-magnetic impurities in such a helical 
liquid. Our results confirm the absence of backscattering from non-magnetic impurities observed 
in recent experiments and predict new interference features, uniquely characteristic of the helical 
liquid, when the scatterers are magnetic. 
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A surface of the three dimensional strong topological 
insulator (STI) [1-3 is a very special place. Topologi- 
cal invariants that characterize the bulk band structure 
of the underlying time-reversal invariant crystal guaran- 
tee the existence of an odd number of gapless surface 
states with the characteristic Dirac dispersion. The sur- 
face electron spins are aligned in the plane of the surface 
and point in the direction perpendicular to the momen- 
tum vector, as indicated in Fig. [l^. Such arrangement 
of electron spins and momenta has been termed 'helical 
liquid' and is predicted to exhibit a number of unusual 
physical properties [¥-^. 

One important property of electrons in the helical liq- 
uid, noted early on [1 , is the absence of backscattering 
in the presence of non-magnetic impurities. Since elec- 
trons with opposite momenta also have opposite spins, 
backscattering requires a spin- flip process, which can- 
not occur in the absence of time-reversal symmetry (T) 
breaking. This fundamental property of the helical liquid 
has been recently tested in a series of experiments [10]- 
[13] using the technique of the Fourier-transform scanning 
tunneling spectroscopy (FT-STS) applied to the previ- 
ously discovered STIs Bii_3,Sb^ and BisTes [HIlS. The 
experimental results are consistent with simple heuristic 
arguments for electron scattering in the helical liquid. 

In this Communication we develop a detailed theory of 
FT-STS in a helical liquid formed on the surface of a STI. 
We focus on the characteristic universal physics encoded 
in the low-energy massless Dirac Hamiltonian (TJ [9] , 
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H = va-{zxp)^Yl aaVa{r) (1) 

describing the surface state of all STIs at probe frequen- 
cies uj tuned sufficiently close to the Dirac point. In Eq. 
([1]) 1' is the Fermi velocity, a = (cri, (72, crs) is the vec- 
tor of Pauli spin matrices, ctq = 1, p = —i{dx-,dy) is the 
planar momentum operator (we take h=l) and V'a(r) is 
the impurity potential in channel a. 

We study the ordinary tunneling and, with an eye 
on the future experiments, we make predictions for the 




FIG. 1: (Color online) a) Dirac dispersion and spin orien- 
tation in helical liquid on the surface of a strong topological 
insulator, b) FT-STS response ^Aoo(q, ^) as a density plot in 
the (qx^Qy) plane (top), cut along the qx direction (bottom). 



spin-resolved FT-STS [16j. We find that non-magnetic 
impurities produce only weak, non-singular response in 
the ordinary FT-STS (Fig. [l]3), consistent with the ex- 
pected absence of backscattering. Interestingly, we find 
that weak magnetic impurities produce no response in the 
ordinary FT-STS, while strong magnetic impurities pro- 
duce only weak, non-singular response, except possibly at 
a resonant frequency. The behavior is even more interest- 
ing for a spin-resolved probe. Here, consistent with time 
reversal symmetry, non-magnetic impurities produce no 
signal, irrespective of their strength. Magnetic impuri- 
ties, on the other hand, give rise to distinctive FT-STS 
patterns with inverse square root singularities at mo- 
menta Iqc^I = 2uj/v. Depending on the direction of the 
impurity magnetic moment and the direction in which 
the probe spin current is polarized we find and classify 
patterns with pronounced two- and four-fold rotational 
modulations. Such modulated patterns, in conjunction 
with future spin-resolved FT-STS experiments, can be 
used to probe the fundamental properties of the helical 
liquid as well as the nature of disorder that underlies 
these patterns. We note that other aspects of magnetic 
impurities on STI surface have been studied in Ref. 0. 
The power of FT-STS technique, originally developed 
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TABLE I: Results for FT-STS response function 27rAQ,^(q, icj) evaluated from Eq. ([7|. Here Q represents the ultraviolet cutoff, 
comparable to the bandwidth of the surface state, z — 2iuj/\c\\ and q = {Qx^qy) /\^- Functions T{z) and Q{z) are defined in 
the text. 
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in the context of simple metals [TT and perfected in 
the studies of high-temperature cuprate superconductors 
[18], lies in its ability to use impurities, present in any 
real material, to probe the electronic response of the un- 
derlying ideal clean material at finite momenta q. The 
experiment measures the local density of states, n(r,cj), 
at a large number (~ 10^) of real-space locations r on 
the sample surface. The spatial Fourier transform of this 
signal n(q, cj) contains the useful information. Theoret- 
ically it can be related to the full electron propagator 
G(r, r';a;) as 



n(q,cj) 



TT 



dVe-^^-^Tr[G(r,r;cj)]. (2) 



Here the trace is taken over spin and ^ denotes the 
strength of a branch cut across the real frequency axis 
^f{uj) = [f{oJ + iS) — f{uj — iS)]/2i^ with S a positive in- 
finitesimal. In momentum space the electron propagator 
has a simple representation in terms of the T matrix, 

G(k,k';w) = 5kk'G°(k,w) + G°(k,w)fkk'MG°(k',w), 

(3) 

where G^(k, cj) = [u — v{kx(Jy — kyCJx)] ^ is the unper- 
turbed propagator and the T matrix is subject to the 
Lippman-Schwinger equation 



Tkk'M = Vk-k' + ^ Vk-qG^(q,a;)Tqk'(a;). (4) 
q 



Matrix Vk = ^^^(3 Fourier transform of the 

impurity term in Hamiltonian ([T]). 

Below, in addition to the ordinary FT-STS we shall 
consider also the spin-resolved FT-STS which is obtained 
by replacing Tr[criG(r, r; a;)] in the integrand of Eq. 
Here and hereafter Greek indices run from to 3 while 
Roman indices run from 1 to 3. It is useful to consider the 
quantity nc./^ (q,cj) which describes the FT-STS response 
of a tunneling probe in charge/spin channel a to weak 
impurities in charge/spin channel (3. 

When the impurity potential is weak it is permissible 
to employ the Born approximation, Tkk'(co') ~ Vk-k'- In 
this limit, as first noted by Capriotti et al. [19], the 
interesting q-dependent part of the FT-STS signal can 
be expressed in a simple factorized form, 



-|Kf|^[A«^(q,c^)], 



(5) 



where 

A,;3(q, ^) = E Tr[a,GO(k, uj)a^G'{k - q, uj)]. (6) 

k 

Since is a Fourier transform of a random potential one 
expects it to be a featureless function of q. Ac,/3(q, cj), on 
the other hand, represents the response of the underlying 
clean system and contains, in general, prominent features 
as a function of q that can be used to directly infer its 
properties. We now evaluate this response function for 
our model Hamiltonian ([T]). Working in the continuum 
limit and passing to the Matsubara frequency we have 



Ac,/3(q, iuj) 



(fk 



L 



af3 



(27r2) [a;2 + A:2][a;2 + (k-q)2 



(7) 



where we also set v = 1 . The factor in the numerator con- 
tains the trace of the Pauli matrices and can be expressed 
as Laf3 = 2r]f3K,iK,jTT[aaCria(3aj], where k, = (kx^ky^iuo)^ 
1^' {kx - Qx, ky - Qy, -iuj) and 7^o,3 = 1, ^1,2 -1. 

Integrals of the type indicated in Eq. ^ are evalu- 
ated most conveniently using the technique of Feynman 
parametrization [20 and have been studied in the context 
high- Tc cuprate superconductors [21] and more recently 
graphene [22 . In fact we may read off the results for 
Aqo and A33 directly from these studies and the remain- 
ing integrals can be evaluated in a similar fashion. Our 
results are summarized in Table I which represents the 
main result of the present study. They are expressed in 
terms of two functions, 



g{z) = 



: arctan 



1 



(8) 



and T{z) — {—z^ — l)S(^) of the dimensionless variable 
z = 2iuj/\q\. The results listed in the Table I satisfy 



Ac/3(q,^) = A/3a(-q,^), 



(9) 



a property that can be easily established from Eq. (|6|. 

The first line in the table {a = 0) describes the or- 
dinary spin-unpolarized FT-STS. The response to non- 
magnetic impurities Aqo, shown in Fig. [T]3, is non- 
vanishing but weak. In the ordinary metal such response 
would entail a singularity at momenta q^^ such that the 
band energy e(qc^/2) = cj, the probe bias. This singu- 
larity arises from impurity scattering between states at 



momenta qu;/2 and — In the present case of he- 
lical liquid such backscattering processes are prohibited 
by time-reversal symmetry [TJ [TQUTSI and consequently 
^Aqo shows only a kink. Interestingly, we find that FT- 
STS response to magnetic impurities^ Aoi, vanishes iden- 
tically in the Born limit. In this case T does not prohibit 
backscattering but the response nevertheless vanishes 
due to the symmetry property of the integral in Eq. 
Specifically, Eq. (|9| implies that Aoi(q, cj) = A^o(— q, co') 
and we shall see below the that latter must vanish due 
to T- invar iance. 

Table I also indicates that response of spin-polarized 
FT-STS to non-magnetic impurities, A^o, vanishes. This 
can be understood as follows. With non-magnetic impu- 
rities the surface of STI remains T-invariant and there- 
fore cannot produce a response in the spin channel. We 
expect this conclusion to remain valid even beyond the 
Born approximation and beyond the simple linear Dirac 
model adopted in this study. 

Spin-polarized FT-STS in the presence of magnetic im- 
purities, Aij, shows the most interesting behavior. In all 
cases ^Aij(q, cj) exhibits an inverse square-root singular- 
ity, contained in the function Q{z) Eq. at momenta 
= 2uj/v. In addition, all channels except for A33, 
show interesting angular dependence on q with two- and 
four-fold symmetry. For a general direction of impurity 
magnetic moment m and the STM tip spin-polarization 
direction fi, the singular part of the response can be writ- 
ten compactly as 



2tt 



(m-Q*)(n-Q)^2^(^), 



(10) 



where Q = {qxAyi |q|/2^cc;) and * denotes complex con- 
jugation. These patterns, displayed in Fig. [2| could be 
used to identify the dominant source of scattering in fu- 
ture spin-resolved FT-STS experiments. Specifically, the 
distinctive angular dependence can be exploited to de- 
duce the direction of the magnetic moment of impurities 
deliberately deposited on the surface of STI [23^. 

Our result of vanishing responses in Aoj and A^o chan- 
nels raises the question of whether this conclusion sur- 
vives beyond the Born approximation. Corrections to 
the Born approximations may become important when 
the scattering potential is strong. To this end we now 
consider the full T matrix governed by Eq. For an 
arbitrary impurity potential it is possible to find the solu- 
tion only numerically. For isolated point-like impurities 
it is however possible to obtain exact analytical results. 
In the following we shall focus on this limit. 

For a single impurity characterized by a (5-function real 
space potential at the origin we have Vk = ^^^(3^ 
dependent of k. The solution for the T matrix is simplest 
and most instructive when we consider scattering in a sin- 
gle channel, Vk = V^cFf^ (no summation over The T 
matrix is then also momentum independent and can be 
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FIG. 2: Spin-resolved FT-STS response ^Aij(q, cj) to mag- 
netic impurities. Top row gives the density plot in the (^x, Qy) 
plane in channel {ij) indicated in the upper right corner. Bot- 
tom row shows the corresponding cuts along the indicated 
lines parametrized by qx. 



written as 



(11) 



Substituting this form into Eq. Q and taking the trace 
we find an equation for T^{uo) of the form 

CFaCFf3CF^]T^{uj), (12) 



where 

5oH = jETr[G°(k,< 



-5— In I 1 
47r 



(13) 

Here S is the area of the STI surface and for finite concen- 
tration of impurities it should be replaced by the inverse 
impurity density n'J^ . The solution of Eq. (Il2| reads 



l-VPgo{oj)' 
and all other components vanish. 



(14) 
(15) 



Eq. ( 14 ) describes the expected resonant enhancement 
of the scattering potential due to the higher order terms 
in the Born series. This will affect the amplitude but not 
the momentum structure of the FT-STS signal. 



The result in Eq. (15) is more interesting and informs 
us that starting at the second order in the Born expan- 
sion scattering potential in the aj channel (i.e. magnetic 
impurities with the moment along direction j) produces 
non-zero T matrix in the charge channel. Magnetic impu- 
rities, therefore, will be visible by the ordinary FT-STS 
probe, although the signal might generally be weak due 
to its appearance in the second order of the expansion in 
the powers of . Since the T matrix does not introduce 
any new momentum dependence for point-like scatterers 
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the momentum-space structure of ^noj(q, cj) will be the 
same as ^noo(q, displayed in Fig, [lb. 

Finally it is worth noting that Eq. ([12| implies T^iuj) — 
0, confirming our previously stated expectation that non- 
magnetic impurities cannot produce a signal in the spin- 
resolved FT-STS. Therefore, (5nio(q, co') remains zero to 
all orders in the Born expansion, as dictated by T- 
invariance of the underlying system. 

Our results presented above underscore the unique ca- 
pabilities of the FT-STS technique and its spin-polarized 
counterpart, applied to the surface of a 3-dimensional 
strong topological insulator. The expected absence of 
backscattering in the helical liquid formed on such a sur- 
face in the presence of non-magnetic impurities J] [TOl - 
[T3] is reflected by the non-singular FT-STS response 
shown in Fig. [l^. This stands in a sharp contrasts to 
the response of the ordinary metal which involves inverse 
square root singularity when e(q/2) = cj. 

In addition to this finding our study reveals several 
striking and unexpected features when the impurities are 
magnetic. First, although backscattering is not prohib- 
ited in this case, we find that the FT-STS response never- 
theless vanishes identically for weak scattering potential 
treated in the Born approximation. The reason for this 
unexpected null result is the symmetry property Eq. ([9| 
of the response function AQ,^(q, uo) which equates the rele- 
vant response Aoj (q, uo) to Ajo(— q, oo). The latter in turn 
underlies the spin-polarized response to weak nonmag- 
netic disorder and must therefore vanish in a T-invariant 
system. We note that this conclusion does not rely in 
any way on our assumption of the Dirac Hamiltonian ([T]) 
but follows from the general requirement of time reversal 
invariance and symmetry ([9|, and will remain valid for 
arbitrary momenta away from the Dirac point. Indeed 
this result is applicable to an arbitrary T-invariant sys- 
tem and indicates that the response of ordinary FT-STS 
to magnetic part of the the scattering potential will gen- 
erally be weak as it appears only in the second order of 
the Born expansion. 

In order to get strong FT-STS response to magnetic 
disorder one must employ the spin-resolved tunneling 
probe [16]. In this case the inverse square root diver- 
gence at e(q/2) = a; is recovered. In our Dirac model 
of STI surface ^ this singular response is accompanied 
by a distinctive pattern of rotational anisotropy summa- 
rized by Eq. ([lO| which appears when either m or n have 
a component in the plane of the sample surface. 

To model the FT-STS response away from the Dirac 
point one must include corrections to the Dirac Hamil- 
tonian reflecting the underlying band structure at larger 
momenta. Very recently, such terms have been consid- 
ered to model the surface state band structure of Bi2Te3 
and results in good agreement with experiments have 
been obtained [24l [25] . For other topological insulators 



such effects will be different as they depend on the de- 
tails of the underlying crystal structure. Our results, on 
the other hand, will remain applicable to any STI sur- 
face when the probe bias is tuned sufficiently close to the 
Dirac point and constitute the universal low-energy the- 
ory of FT-STS on the topologically protected STI surface 
states. 
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